In a conventional serial processing computer (Neiman computer), it is very difficult to adjust data processing capabilities with the change of usage and environment. Accordingly, more flexible data processing units have been demanded and developed to process data using a parallel and distributed method in a new hierarchical network configuration specifically in the technical field of pattern recognition, applicable filter, etc. In this hierarchical network data processing unit, programs are not written explicitly, but an output signal (output pattern) provided by a network structure in response to an input signal (input pattern) presented for learning determines a weight value of an internal connection of a hierarchical network structure so that the output signal corresponds to a teaching signal (teacher pattern) according to a predetermined learning algorithm. When the weight value is determined by the above described learning process, a "flexible" data processing function can be realized such that the hierarchical network structure outputs a probable output signal even though an unexpected input signal is inputted.
In the data processing unit in this hierarchical network configuration, there is an advantage in that a weight value of an internal connection can be automatically determined only if a learning signal is generated. However, there is also a disadvantage in that the content of a data process which is dependent on a weight value is not comprehensible. A successful solution would allow this data processing unit in a network structure to be put into practical use. To practically control this data processing unit in a hierarchical network configuration, an effective means must be presented for high speed, high precision data processing.
In a data processing unit comprising a hierarchical network configuration, a hierarchical network comprises one kind of node called a "basic unit" and an internal connection having a weight value corresponding to an internal state value. FIG. 1 shows a basic configuration of a basic unit 1. The basic unit 1 is a multi-input/output (i/o) system. That is, it comprises a multiplier 2 for multiplying a plurality of inputs by respective weight values of an internal connection, an accumulator 3 for adding all products of the above described multiplications performed by the multiplier, and a function converter 4 for applying function conversion such as a non-linear threshold process to the resultant accumulation value so that a final output can be obtained.
Assuming that a layer h is a preprocess layer and a layer i is a post-process layer, the accumulator 3 of the i-th basic unit 1 in the layer i executes the following operation using expression (1), and the function converter 4 executes a threshold operation using expression (2). EQU x.sub.pi =.SIGMA..sub.h y.sub.ph W.sub.ih ( 1) EQU y.sub.pi =1/(1+exp (-x.sub.pi +.theta..sub.i)) (2)
where
h: a unit number in the layer h PA1 p: a pattern number of an input signal PA1 .theta..sub.1 : a threshold of the i-th unit in the layer i PA1 W.sub.ih : a weight value of an internal connection between the layers h and i Y.sub.ph : output from the h-th unit in the layer h in response to an input signal in the p-th pattern. PA1 "if (X is small) and (Y is small) then Z is middle," the connection of the unit 22b, 22c, and 22e respectively to the unit 23a is not required. Thus, the pre-wired-rule-part neuro is defined as a data processing system where only necessary parts are connected between the antecedent membership function realizer 26 and the rule part 17 (in the rule part 17) and between the rule part 17 and the consequent membership function realizer, according to the rules of a fuzzy teacher.
The data processing unit in a hierarchical network configuration configures its hierarchical network such that a plurality of the above described basic units 1 are hierarchically connected as shown in FIG. 2 (with input signal values distributed and outputted as an input layer 1') thus performing a parallel data process by converting input signals into corresponding output signals.
The data processing unit in a hierarchical network configuration requires obtaining, by a learning process, a weight value of a hierarchical network structure which determines the data conversion. Specifically, a back propagation method attracts special attention for its practicality as an algorithm of the learning process. In the back propagation method, a learning process is performed by automatically adjusting a weight value W.sub.ih and a threshold .theta..sub.i through feedback of an error. As indicated by expressions (1) and (2), the weight value W.sub.ih and the threshold .theta..sub.i must be adjusted simultaneously, but the adjustment is very difficult because these values must be carefully balanced. Therefore, the threshold .theta..sub.i is included in the weight value W.sub.ih by providing a unit for constantly outputting "1" in the layer h on the input side and assigning the threshold .theta..sub.i as a weight value to the output, therefore allowing it to be processed as a part of the weight value. Thus, expressions (1) and (2) are represented as follows: EQU x.sub.pi =.SIGMA..sub.h y.sub.ph W.sub.ih ( 3) EQU y.sub.pi =1/(1+exp (-x.sub.pi)) (4)
In the back propagation method, as shown in the three-layer structure comprising a layer h, a layer i and a layer j in FIG. 2, the difference (d.sub.pj -y.sub.pj) between an output signal y.sub.pj and a teaching signal d.sub.pj is calculated when the output signal y.sub.pj outputted from the output layer in response to an input signal presented for learning and a teaching signal d.sub.pj for matching the output signal y.sub.pj are given. Then, the following operation is performed: EQU .alpha..sub.pj =y.sub.pj (1-y.sub.pj) (d.sub.pj -y.sub.pj) (5)
followed by: ##EQU1##
Thus, an updated weight value .DELTA.W.sub.ji (t) between the layers i and j is calculated, where t indicates the count of learnings.
Then, using the resultant .alpha..sub.pj, the following operation is performed: ##EQU2## followed by: ##EQU3##
Thus, an updated weight value .DELTA.W.sub.ih (t) between the layers h and i is calculated.
Then, weight values are determined for the following update cycles according to the updated value calculated as described above: ##EQU4##
By repeating the procedure above, learning is completed when the weight values W.sub.ji and W.sub.ih are obtained where an output signal y.sub.pj outputted from the output layer in response to an input signal presented for learning corresponds to a teaching signal d.sub.pj, a target of the output signal y.sub.pj.
When the hierarchical network has a four-layer configuration comprising layers g, h, i, and j, the following operation is performed: ##EQU5##
followed by: ##EQU6##
Thus, the updated amount .DELTA.W.sub.hg (t) of a weight value between the layers g and h can be calculated. That is, an updated amount .DELTA.W of a weight value between the preceding layers can be determined from the value obtained at the last step at the output side and the network output data.
If the function converter 4 of the basic unit 1 performs linear conversion, the expression (5) above is represented as follows: EQU .alpha..sub.pj =(d.sub.pj -y.sub.pj) (11)
the expression (7) above is represented as follows: ##EQU7## and the expression (9) above is represented as follows: ##EQU8##
Thus, an expected teaching signal is outputted from the output layer in response to an input signal presented for learning in the data processing unit in a hierarchical network configuration by assigning a learned weight value to an internal connection in the hierarchical network. Therefore, a data processing function can be realized such that the hierarchical network structure outputs a probable output signal even though an unexpected input signal is inputted.
It is certain that, in the data processing unit in a hierarchical network configuration, data can be appropriately converted with a desirable input-output function, and a more precise weight value of an internal connection can be mechanically learned if an additional learning signal is provided. However, there is also a problem in that the content of data conversion executed in the hierarchical network structure is not comprehensible, and that an output signal cannot be provided in response to data other than a learning signal. Therefore, an operator feels emotionally unstable when data are controlled, even in the normal operation of the data processing unit in a hierarchical network configuration, because an abnormal condition is very hard to properly correct. Furthermore, as a learning signal is indispensable for establishing a data processing unit in a hierarchical network, a desired data processing function may not be realized when sufficient learning signals cannot be provided.
On the other hand, "a fuzzy controller" has been developed and put into practical use recently for control targets which are difficult to model. A fuzzy controller controls data after calculating the extent of the controlling operation from a detected control state value by representing, in the if-then form, a control algorithm comprising ambiguity (such as determination of a human being) and executing this control algorithm based on a fuzzy presumption. A fuzzy presumption enables the establishment of an executable teacher for use with a complicated data processing function by grouping the combinations of input/output signals and connecting them ambiguously according to attribute information called "a membership relation." A fuzzy teacher generated by the fuzzy presumption has an advantage in that it is comparatively comprehensible, but has a difficult problem in that a value of a membership function cannot be determined precisely, and the exact relation of the connection between membership functions cannot be determined mechanically, thus requiring enormous labor and time to put desired data processing capabilities into practical use.